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PREFACE 


This  paper  vas  prepared  from  an  hour  talk  given  to  the  secondary 
cicthcciatics  teachers  of  Pasadena  City  Schools  on  January  1962.  The  tallc 
vas  part  of  the  Matheraatics  Symposiums  for  IS6I-I962  planned  by  W.  G.  Norris, 
Mathematics  Supervisor  of  the  Pasadena  School  System. 


UNK/U^  PROGH/J'SCIIG 


Richard  C .  Kao* 

The  RAIID  Corporation,  Santa  Monica,  California 

1.  Historical  Notes 

Linear  progrci.iming  is  a  relatively  recent  subject  in  nsthcnatics,  having 
devclo])od  mostly  since  when  the  Air  Force  set  up  Project  SCOOP 

(Scientific  Caaputation  of  OptlnLLn  Programs)  to  investigate  efficient  organi¬ 
sation  of  large-scale  programming  and  scheduling  activities.  Since  that  time 
^ho  subject  has  enjoyed  a  flourishing  growth,  both  in  theory  and  in  applica- 
cions.  Most  of  the  early  work  was  done  in  the  Air  Force,  which  has  also  lent 
generous  support  to  sponsoring  various  symposia  on  the  subject. 

Tne  earliest  work  known  of  the  linear  programming  type  was  a  transporta¬ 
tion  problem  posed  by  Hitchcock  in  19^1.  In  the  same  year  a  diet  problem  was 
considered  by  Cornfield,  ^rtiich  was  studied  more  intensively  by  Stigler  in  19^^. 
Parallel  to  these  spotty  early  efforts,  Leontief  carried  on  input-output  indus¬ 
try  studies  since  the  early  ’30s  which  later  proved  to  be  related  to  linear 
programming  problems.  Project  SCOOP  conpriscd  most  of  the  original  contribu¬ 
tors  to  the  subject,  including  inter  alia  George  Dantzig,  Marshall  Wood,  Murray 
Geisler,  Leon  Goldstein,  Julian  Holley,  Walter  Jacobs,  Alex  Arden,  and  M;iil 
Schell.  A  great  forward  leap  was  made  when  this  group  joined  forces  v;ith  two 
other  groups:  Koopmans  at  the  Cowles  Commission,  who  had  done  much  inde¬ 
pendent  work  in  activity  analysis  of  the  shipping  industry,  and  von  Neaniarui  and 
Tucker  at  Princeton  with  their  students  (e.g. ,  Gale,  Kuhn,  Goldman  and  Gomory). 

One  unique  feature  of  linear  programming  as  a  subject  in  applied  mathe¬ 
matics  is  its  rapid  growth  in  applications  —  to  problems  in  goveranent, 
industry  and  business  --  pari  passu  with  its  theoretical  developnent.  Tliia 

♦Any  views  exnressed  in  this  paper  are  those  of  the  author.  They  snould 
not  be  interpreted  as  reflecting  the  views  of  The  RAI®  Corporation  or  the  of¬ 
ficial  opinion  or  policy  of  any  of  its  governmental  or  private  research  sponsors. 
Papers  ore  reproduced  by  The  Corporation  as  a  courtesy  to  members  of  its 

staff. 
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(2.1) 


simultaneous  evolution  of  the  subject  along  tvo  onts  is  generally  the 
exception  rather  than  the  rule  in  the  history  of  mathematics. 

2.  The  Optimal  Diet  Problaa 

Linear  programnlng  deals  vith  maximization  (or  minimization)  of  a 
linear  function  of  n  veiriables  where  the  variables  are  subject  to  a  set  of 
linear  constraints  (l.e.^  equations  or  Inequalities).  All  or  seme  of  these 
variables  may  further  be  required  to  be  nonnegative.  Let  us  illustrate  the 
problem  with  a  simple  exonple. 

Suxipose  ve  have  four  foods:  corned  beef,  cabbage,  potatoes,  and  mlUc, 
and  wish  to  prepare  a  wholesome  diet  fran  them.  To  do  this  we  ask  ourselves 
first  the  question:  What  is  it  that's  so  important  in  a  diet?  The  obvlou* 
answer  is  its  nutritive  value.  So,  let  us  simplify  the  problem  by  con¬ 
sidering  only  three  classes  of  nutrients:  protein,  carbohydrates  and 
vitamins;  and  the  nutrient  contents  of  the  four  foods  are  given  as  follows: 


Corned  Beef 

Cabbage 

Potatoes 

Milk 

7i 

^2 

73 

Ih 

Proteins 

6 

0 

1 

3 

*^1 

Carbohydrates 

^2 

2 

2 

k 

1 

r 

2 

Vitamins 

1 

3 

1 

2 

^1 

^2 

^3 

^4 

where  "6"  means  that  1  unit  of  corned  beef  contains  6  units  of  proteins, 
and  that  1  unit  of  potatoes  contains  4  units  of  carbohydrates,  etc. 

We  assume,  of  course,  that  all  these  units  have  been  proi)crly  defined,  say 


by  the  nutritionists.  Let  us  call  (2.1)  the  nutrition  matrix.  A  diet 
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(or  nenu)  Is  any  cocitination  of  the  four  foods,  sey  in  amounts  of 
lit-  For  any  given  diet  (  7j^,  7g,  7^,  ), 


(2-2)  6  */l  +  0-  *  1-  73  *  3-7i, 

is  the  toted  amount  of  protein,  and 


(2.3)  271*272  +  1.73.1.7^ 

is  the  total  amount  of  carbohydrates.  Nov  we  wish  to  Impose  ein  additional 
restriction  on  our  diet  that  it  provides  at  least  units  of  protein, 

units  of  carbohydrates  and  units  of  vitamins.  Such  a  diet  will  be 

called  a  feasible  diet.  More  abstractly,  let  a^^j  denote  the  units  of 
nutrient  N,  (i^l, •••,  m)  contained  in  one  unit  of  food  F.  (j«l, •••,  n). 

A  J 

Then  a  feasible  diet  is  a  set  of  n  numbers  satisfying 

(2.4)  ®  J  n 

and 


(2.5) 


•  •  • 


r 

i 


i  B  1,  *  •  •  /  “ 


As  yet  no  maximization  or  minimization  is  involved,  euid  it  is  obvious  that 

(2.5)  may  generally  be  satisfied  by  making  the  large.  We  are  there¬ 

fore  interested  not  in  eating  any  arbitrarily  large  diet  (i.e.,  large 

but  one  which  will  yield  the  least  cost 
J 

(2.6)  + ...  * 


where  13.  (3*1,  •••,  n)  is  the  given  unit  price  of  F.,  and  consequently  (2.6) 

J  V 
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is  the  total  price  to  be  paid  for  the  diet  (  7n^*  ^  feaciblc  diet 

minimizing  (2.6)  is  called  an  optimal  diet.  CXir  problem  is,  in  sammory, 
the  foHoving:  Among  all  feasible  diets,  to  find  one  which  io  optimal. 

This  is  the  optimal  diet  problem  which  gives  rise  to  the  standard  minimum 
problem  of  the  linear  programming  tyi>e. 

To  each  standard  minimum  problem  corresponds  a  second  linear  program¬ 
ming  problem,  callc-d  the  dual  problem.  Let  us  first  state  it  formally  and 
then  try  to  explain  it.  Wiat  problem  is  to  find  numbers  (i--l, *•*,  m) 
satisfying 

(2.7)  ^i  ■  ^  i  «!,•••>  m 


and 


(2.8) 


i 


a. 


1  "10 


< 


such  that 

(2-9) 


+ 


i  r 

m  m 


j  s  !>•••>  n 


is  maximized.  The  symmetry  between  (2.U)  to  (2.6)  and  (2.?)  to  (2.9)  In 
lanedlately  apparent,  and  this  duality  occupies  the  central  place  in  the 
whole  subject  called  linear  progranzaing. 

We  now  give  a  heuristic  explanation  of  the  dual  problem.  Let  us 
begin  with  (2.9)»  Since  (i*»l, •«•,  m)  are  units  of  nutrients 

(2.9)  makes  sense  only  if  denotes  monetary  value  per  unit  of  so 

that  the  simmation  in  (2.9)  niay  be  performed  in  the  common  denominator: 
money.  (2.?)  merely  states  that  these  unit  monetary  values  must  be  non- 
negative  and  (2.8)  states  that  the  sum  total  of  unit  monetary  values  over 
all  nutrients  must  not  exceed  the  unit  selling  price  of  Fy  Hence, 
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Is  Just  the  unit  cost  of  and  (2.8)  says  that  the  sum  total  of 

nutrients  going  into  one  unit  of  F.  must  not  cost  more  than  the  selling 

J 

price  of  F..  Subject  to  this  restriction,  ve  are  to  choose  that  set  of 

J 

^^'s  maximized  (2.9)*  Therefore,  the  dual  problem  is  a  meaningful  eco- 
ncmlc  problem  facing  the  resource  (i.e.,  nutrient)  owners.  They  wish  to 
set  the  hipest  possible  costs  on  the  resources  subject  to  the  condition 
that  the  food  manufacturer  may  still  continue  to  produce.  The  fundsmental 
theorem  of  linear  programming  states  that  if  feasible  solutions  exist  to 
the  primal  problem  and  its  dual,  then  necessarily  optimal  solutions  exist 
for  both  such  that  the  minimum  of  (2.6)  equals  the  maximum  of  (2.9)* 

This  last  condition  leaves  the  food  manufacturer  with  zero  profit  and  is 
sometimes  called  the  equilibrium  condition  under  pure  competition.  More¬ 
over,  the  connection  between  linear  programming  and  game  theory  is  also 
suggested. 

3.  Linear  Systems  and  Linear  Programming 

Basic  to  the  study  of  linesu*  progracming  is  the  theory  of  linear 
systems  (i.e.,  equations  and/or  inequalities).  Generally  speaking,  there 
ore  two  separate  problems  analogous  to  those  in  the  study  of  linear 
equations,  that  is,  linear  algebra.  The  first  deals  with  the  existence 
(or  non-existence)  of  solutions  to  a  system  of  linear  inequalities,  and 
the  second  deals  with  the  structure  of  the  solution  set.  There  is  a 
beautiful  algebraic  theory  of  linear  inequalities  which  tells  us  when 
solutions  will  or  will  not  exist.  In  this  theory  we  see  a  natural  pair 
of  linear  systems  such  that  if  one  has  no  solution,  the  other  must  have, 
and  conversely.  A  general  rule  for  finding  this  pair  of  linear  systems 
is  that  the  variables  in  one  correspond  to  the  constraints  in  the  other. 
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and  vice  versa.  More  specifically,  any  nonrestricted  variable  in  one 
system  corresponds  to  a  linear  equation  in  the  other  system,  whereas  a 
restricted  (i.e.,  nonnegative  or  nonpositive)  variable  in  one  system 
corresponds  to  a  linear  inequality  in  the  other  system.  This  pair  of 
linear  systems  is  also  related  to  orthogonal  complements  in  the  theory  of 
vector  spaces. 

The  structure  of  the  solution  set  of  a  linear  system  is  best  studied 
geometrically  by  means  of  the  theory  of  convex  sets.  Here  we  see  that  the 
solution  set  of  a  homogeneous  linear  system  is  a  convex  cone  and  that  of 
a  nonhomogeneous  linear  system  a  convex  polytope.  In  either  case,  a 
finite  set  of  extreme  points  or  vectors  exist  such  that  all  other  solutions 
are  convex  linear  combinations  (i.e.,  centers  of  gravity  with  varying 
veldts)  of  these  extreme  solutions.  Tbe  solution  set  of  the  most  general 
linear  system  is  the  (vector)  em  of  a  convex  cone  and  a  convex  polytoi>e. 

From  the  theory  of  linear  systems  follows  immediately  the  fundamental 
theorem  of  lineeu:  prograntaing.  In  terms  of  matrix  gemes,  this  is  also 
called  the  mlnimax  theorem  first  proved  by  John  von  Neumann  in  1926.  The 
entire  theory  may  be  extended  to  nonlinear  (say  convex  or  concave)  systems, 
which  must,  however,  have  some  global  property.  Such  extension  may  be 
found  in  connection  with  Fenchel' s  work. 
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Caaputatlonal  .^fethods 

There  is  a  separate  body  of  theories  related  to  the  computational 
aspect  of  linear  programning,  the  first  of  which  was  the  elegant  simplex 
algorithm  by  Dantzig.  Let  us  ii-Lustrate  it  by  considering  the  following 
problem:  To  find  =  0  (j=l,  •••,  n)  minimizing 

fi.V.  +  ...  +  fi  rj 

1  '  1  n  »  n 


subject  to 


(4.2) 


“11  ^  •••  “tain  '  ^1 

•  •  • 

“mill  *  •••  “mnin  “ 


In  (4.2)  we  include  equations  only.  If  inequalities  are  involved,  v;e 
merely  Introduce  additional  variables  to  change  them  into  equations.  These 
new  variables,  one  for  each  inequality,  will  be  given  zero  coefficients 
in  (4.1). 

There  ore  two  steps  to  the  simplex  algorithm: 

1.  We  must  find  a  first  feasible  solution,  and 

2.  Given  that  a  first  feasible  solution  is  found,  v/e  arc  to 
find  another  feasible  solution  which  yields  a  possibly  smaller  value  for 
(4.1).  Step  2  must  also  tell  us  when  to  stop  looking  for  a  better  solution. 

We  discuss  Step  2  first  and  assume  a  first  feasible  solution  exists. 

Let  the  first  p  columns  of  coefficients  on  the  left  side  of  (4.2)  bo 

( linear IjO  independent  and  the  ranaining  dependent  on  these.  Then  a  work 
sheet  of  the  following  type  can  be  set  up. 
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ric^it-hamd 


col.  1 

•  •  « 

col.  p 

col.  (iH-l) 

•  •  • 

col.  n 

side 

L 

a  •  •  • 

r 

a 

p 

... 

p+1 

a  •  •  • 

s 

a 

n 

•  •  •  Q 

c 

1 

a 

t 

• 

« 

rH  C 

0 

...  Q 

T  »  •  • 

l,p+l 

# 

t 

Is 

0 

0  0^^  000 

In 

7a 

• 

• 

• 

r 

a 

• 

0 

• 

0 

$ 

0  0  0 

0 

0 

0 

1 

0 

• 

0 

0 

0 

0 

0 

0 

^  0  0  0 
r,pfl 

# 

0 

0 

X 

rs 

• 

• 

• 

*  • »  X  ... 

rn 

• 

0 

0 

0 

0 

0 

$ 

0 

• 

• 

0 

0 

0 

0 

0 

0  •  •  • 

0 

•  •  •  X 

t  ,  '  " 

p,IHl 

t 

ps 

•  •  •  X  ... 

pn 

If 

In  this  simplex  tableau  we  express  •  •  • ,  a^  and  c  as  linear  conbina- 

tionp,  of  a^,  •••,  a^.  It  is  instructive  to  recall  our  diet  problem,  in  which 

the  columns  represent  foods.  Then  (4.3)  says  we  can  choose  p  basic 
X  p 

a**", •••,  a^,  then  form  linear  combinations  of  these  to  get  nutritionally 
equivalent  substitutes  of  the  remaining  foods.  For  example, 


ih.’,) 


(4.6) 


(4.4) 


T  1 

I  a  '  +  •  • 
Is 


ps 


that  is,  one  unit  of  food  a  is  nutritionally  equivalent  to  units  of 

X  2  p 

a  plus  units  of  a  plus  so  on  until  X  units  of  a^.  llie  last 

2$  ^  ps 

column  in  (4,3)  simply  states  the  required  nutrient  vector  (  >  ®^) 

may  be  exactly  satisfied  by  units  of  a^, •••,  units  of  a^,  the  first 
feasible  solution  which  we  assume  to  exist.  It  is  interesting  to  note  also 
that  once  a^,  ••*,  a^  arc  chosen  to  be  the  basic  foods,  the  substitutes 
which  can  be  formed  for  them  are  Just  themselves,  i.e.,  use  a^  as  substitute 
for  a^,  *  *  *  ,  eP  as  substitute  for  a^.  In  other  words,  for  basic  foods, 

there  is  no  need  to  find  substitutes,  or  the  substitutes  are  identical  with 


(4.7) 
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the  foods  themselves,  whereas  for  nonbasic  foods,  it  is  possible  to  find 
substitutes  for  them  using  combinations  of  the  basic  foods. 

Our  first  menu  therefore  consists  of  units  of  a^,  •••,  units 

of  and  nothing  else,  for  which  the  total  price  is 


P 


There  may  be  sene  nonbasic  foods  not  yet  in  the  menu  which  would  be  cheaper 
than  their  substitutes  in  terms  of  the  basic  foods.  Take,  for  example, 
a  with  unit  price  fi  .  (^.^)  states  that  in  order  to  find  some  combination 

G 

X  TD  S 

of  a  ,  •  •  • ,  a'^  to  be  nutritionally  equivalent  to  a  ,  we  must  use  uniuS 

of  a"^,  •••,  t  units  of  a^  so  that  the  substitute  would  cost 
*  *  ps 


r  /3  +...+  x  ft  ^  ^ 

Is  '  1  ps  '  p  's 

If  fte  <  ^s,  then  we  would  save  money  by  introducing  a®  into  the  menu. 

But  each  unit  of  a  introduced  into  the  menu  will  displace  units  of 
a^,  •  •  • ,  units  of  a^.  We  want  to  find  the  largest  possible  number  of 

g 

units  1  of  a  which  could  be  introduced  into  the  menu. 

•s 

It  is  certainly  obvious  that  each  additional  unit  of  a®  introduced 
would  enable  us  to  cut  down  on  the  amounts  of  a^,  •  •  • ,  a^  used  in  the  first 

g 

menu.  In  fact,  if  we  use  units  of  a  ,  we  reduce  the  amounts  needed 
for  a^, • • • ,  a^  to 


7  =  7  -  T  7 

'l  'l  Is  ' 


7  ^  3  ^7  —  7 

/p  »p  ps  ' 
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These  quantities  must  still  aremain  nonnegative.  This  condition  will 

certainly  be  fulfilled  if  "T.  “0  since  would  then  not  be  smaller  than 

7^  which  is  nonnegative  to  begin  with.  Hence,  we  need  to  worry  about  those 

cases  in  which  >  0.  Among  these,  we  choose  so  large  as  to  reduce  one  (^. 

r 

to  zero,  that  is,  we  delete  one  of  the  old  foods,  say  a  ,  fran  the  menu 

Q 

completely  and  replace  it  by  a  .  Our  algorithm  for  finding  a  cheaper 
feasible  diet  consists  of  two  steps. 

a 

1.  Introduce  any  nonbasic  food  a  into  the  menu  which  is  cheaper 
than  its  substitute  in  terms  of  the  basic  foods  a^, • • • ,  a^. 

That  is,  if 

(U.8)  /3  <  b  r  4  , , ,  4.  T 

'  '  s  Is  ps  P 

2.  Delete  a  basic  food  a  (1  «  r  *»  p)  from  the  menu  by  intro¬ 
ducing  7®  units  of  a°  where 


(4.9) 


Trs 


7C 

rain  "7“ 
i 


>  0 


These  78  units  of  a®  will  reduce  the  amoimts  used  of  a^,  •••,  a^  from 
7^^, ••*,  7p  7i^***»  7p  defined  by  (4.7).  (  7^  =  0,  of  course,  by 

choice  of  78*) 

After  these  two  steps  we  will  hare  a  new  feasible  solution.  It  is 
easy  to  calculate  the  total  price  for  the  new  menu: 


(4.] 


(4.1 


(4.10) 


•  •  • 


8 
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ThiG  is  lesG  than  or  equal  to  (^.5)  cince 


7 


/ 

r  =  0  and  so 


4.11) 


^lll^  •  •  •  +  /^s76  =  ^1(7i  -  ^Is  72)  •♦••*•+  (  7p  " 

+  PbIb  =  ^l7l  +  ...  +  ^p7p  -  (^1  "^Is  +  •••  +  p  ^ps)  7®  /^S  7c 

=  ^ih  +  •••  ^  ^pTp  ■  7® 


The  last  term  in  (U.ll)  shows  that  the  price  of  the  new  nenu  is  the  dif¬ 
ference  between  the  price  of  the  old  menu  and  the  amount  of  possible  saving 
by  introducing  jg  units  of  a  .  We  repeat  the  process  until  no  nonbasic 
food  can  be  found  which  will  be  cheaper  than  their  substitutes. 

The  foregoing  discussion  gives  the  essence  of  the  simplex  algor: 
and  its  extensions.  We  conclude  this  section  with  a  brief  remark  on  hov  to 
find  a  first  feasible  solution.  To  do  that,  we  consider  the  fo-Uowing 


auxiliary  linear  programming  problem: 
(i=l, •••,  m)  minimizing 


To  find  y.=0  (J=l;--*,  =  0 

J 


+  .12) 


O 

m 


subject  to 

^11  7l  +  •  *  •  ^Jn. 

k13)  ^11  "71  =  y'x 


where  we  may  assume  without  loss  of  generality  =  0  (i=l,***,  i.). 
If  this  minimization  problan  yields  zero  for  (4.12),  then  all  sluc..r 
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(4.13)  vanish  and  the  corresponding  f  will  form  the  first 

feasible  solution  to  our  original  system,  i.e.  (4.2).  If  some  slack 
remains,  then  (4.2)  has  no  feasible  solution.  The  problem  of  finding  a 
first  feasible  solution  to  (4.2)  has  now  been  changed  to  one  of  finding  a 
first  feasible  solution  to  (4.13).  But  a  first  feasible  solution  to 
(4.13)  can  be  found  by  inspection,  namely  by  setting 

There  are  many  variations  as  well,  as  extensions  of  the  simplex 
algorithm,  and  also  other  algorithms  which  are  available  for  canputing 
large-scale  linear  programing  problems.  For  most  general-purpose 
electronic  computers,  computer  programs  now  exist  for  such  use.  We 
me.ition  only  a  few  as  follows:  Burrou^is  Datatron,  Ferranti  Limited 
(English)  Pegasus,  IBM  65O,  701,  704,  705,  709,  1620,  707O,  7090. 

Spen:7  Rand  Univac  1,  IIO3,  n03A. 

Applications  and  Related  Subjects 

We  shall  mention  only  briefly  some  fields  of  application  of  linear 
programming.  As  it  was  noted  in  Section  1,  the  subject  arose  originally 
from  practical  problems  in  transportation,  dietetics  and  military  planning. 
A  rather  comprehensive  survey  of  linear  programming  applications  may  be 
found  in  the  Bibliography  on  Linear  Programming  and  Related  Techniques 
by  Vera  Riley  and  S.  I.  Gass  (Johns  Hopkins  Press,  195Q)* 

The  following  outline  includes  only  some  selected  applications  for 


illustration  purposes: 
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1.  Acricultural  applications: 

a.  Farm  manacenent 

b.  Fecd-mixinc 

c.  Crop  rotation 

2.  Industrial  applications: 

a.  Chemical  industry 

b.  Coiil  Industry 

c.  Iron  and  steel  industiv 

d.  Paper  industry 

e.  Petroleum  industry 

3.  Cocmercial  applications: 

a.  Airline  routing 

b.  Coomunication  networks 

c.  Railway  freight 

d.  Securities  selection 

e.  Inventory  control 

k.  Military  applications: 

a.  Weapon  selection 

b.  Procrar,  planning 

c.  Personnel  assignaent 

Hot  only  has  the  range  of  application  of  linear  programming  croadened 
in  recent  years,  the  subject  has  been  found  related  to  many  other  branches 
of  mathematics.  Of  these,  we  list  a  few: 

l.  Game  theory 

2.  rjetwork  flow  or  circuit  theory' 


3.  Graph  theory 
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4 .  ’^»jinber  theory 

5.  Mathematical  statistics 
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